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METHOD AND APPARATUS FOR PRODUCING A COORDINATED GROUP 

ANIMATION BY MEANS OF OPTIMUM STATE FEEDBACK. 

AND ENTERTAINMENT APPARATUS USING THE SAME 

BACKGROUND OF THE INVENTION 
1. Field of the Invention 

The present Invention relates to a method and 
apparatus for producing an animation to represent in a 
quasi fashion the movement of a group including a 
10 pluxf^lity of members such as persons or animals, and 
more particularly, to a method and apparatus for 
controlling the movements of members as a group. The 
present invention also relates to an entertainment 
appeiratus using such a method. 
15 2. Eiescrlption of the Related Art 

Conventional computer graphic techniques can be 
classified into two categories: group animation and 
physically based animation. 

Physically based animation refers to an animation 
20 which is produced in accordance with a physical 
principle such as a law of mechanics. More 
specifically, equations of motion (simultaneous 
nonljinear equations or differential equations) is 
determined according to Newton's law or Lagrange's law 
25 for jeach member. However, the cost of precisely 
simulating mechanical or physical motion is high. 



1 



0 
to 

1.^ 



.1^ 

7 

ii 

p. 

a :: 



performed 



not 
but 



SC1626 

Therefore • In the art of information processing 
apparatuses, such as computers « or in the art of 
entertainment apparatuses, to which the present 
invention is applied « some simplification is generally 
to reduce the cost. 
That is, in these fields of technologies, it is 
necessary to build a physically accurate model, 
equations of motion based on a quasi-physical 
modejl may be employed if the equations allow the 
10 . motion of the respective members of interest to be 
determined in real time and if the resultant motion 
appears sufficiently natural. 

(1) Group Animation to Represent Group Behavior 
15 The research work by Reynolds has bad a profound 

impact upon the art (C. W. Reynolds, "Flocks, Herds, 
and Schools: A Distributed Behavioral Model", 
Procjeedings of Siggraph 87, Jul,, 1987, 25-34). After 
his research work« animation of a group of birds, 
20 fishes, or animals can be realized by means of 

programming independent members of the group. By 

adjusting parameters associated with the behavior of 

I 

eacli( member, it is possible to control the behavior in 
a discrete fashion. However, when the group is viewed 
25 as at whole, most of the group behavior occurs 
spontaneously. 



.1' t 



• 



St! 5 

W 



p 



SC1626 

Tu and Terzopoulos have produced a more realistic 
aniiratlon of a group Including a plurality of members 
by Improving the reality of the group and the senses 
by sjimulating biomeohanical behavior (X. Tu and D. 
5 Teri^opoulos , "Artificial Fishes s Physics. Locomotion, 
Perception, and Behavior", Proceedings of Siggraph 94, 
Jul J, 1994, 24-29). This pioneering research work 
indljcated that the behavior of a group of fishes or 
anir^als can be synthesized in a more natural fashion 
10 by elmploying a combination of stimulus -response rules. 

I Receiving a stimulus from artificial intelligence 

I 

technology, Funge, Tu, and Terzopoulos have proposed 
and demonstrated a very elegant framework which allows 
the representation of perceptive behaviors of members 

15' of a group (J« Funge, X. Tu, and D. Terzopoulos, 

"Cognitive Modeling: Knowledge, Reasoning and Planning 
for .Intelligent Characters", Proceedings of Siggraph 
99, jjul. 1994, 29-36). They have attained their 
objejctive in that artificial life and the behavior 

20 therieof are animated « 
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(2) ioynamic and Physical Group Animation 

I Dynamic and physical animation has great problems 

in sjtability and controllability. The problems become 

i 

seriious when groups include a large number of members. 
The motion of group members is very complicated, and 
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it has been thought that it is vary difficult to 

i 

mat ijamat leal ly represent the complicated motion. 
Howejver. two physicists « Toner and Tu, have recently 

I 
I 

exp^jaindd using a mathematical model how coordinated 
6 group behavior occurs spontaneously (J. Toner and Y. 
Tu, "Flocks, herds, and schools: A Quantitative Theory 
of f*loc]clng" , Physical Review Letters, Vol. 58, 1999). 
A pitof ound understanding of the motion of a group 

including a plurality of members is important. In 

t 

10 . part^icular, in the art of computer animation, the 

development a high-efficiency method of dynamically 

i 

conlsrolling a group including a plurality of members 
is Required. 

iBrogan and Hodgins have investigated the behavior 

I. 

15 of a| group with significant dynamics and have 

presjented their opinion about the complexity of the 

protjlem (D. Brogan and J. Hodgins, Group Behaviors 

1 

f or jSystems with Significant Dynamics", Proceedings of 
the il995 lEEE/RSJ International Conference on 

20 Intelligent Robots and Systems, Vol. 4, 528-534; D. 
Brogjan and J. Hodgins, "Group Behaviors for Systems 
witlx Significant Dynamics", Autonomous Robots 4(1), 
199?;, pp. 137-153). That is, they have pointed out 
that! motion o£ individual member.s and local 

25 intelxactiona make it difficult to control the group as 
a whole and that a combination of local knowledge and 
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global knowledge is required to efficiently control 
the jindlvldual members. 

iThe objective of the research work by Brogan.and 
Hodglins was not to provide a technique for animation 
5 but ito provide a technique which may be applied to the 
art jof robotics in which realistic equations of motion 

can jbe employed so that a realistic group behavior is 

t 

represented by means of dynamic control. Therefore, 
this! technique is not assumed to be applied to 
10 . technical fields such as animation in which it is 
H reqvt'lred to simplify the motion of the Individual 

members, depending upon the specific purpose. 

I'u i 

; Furthermore, in the conventional techniques 

«E i 

described above, control factors are adjusted 

15 expeirlmentally and manually, and automatic adjustment 

P of t'he control factors has not yet been achieved. 
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SUMMARY OF THE INVENTION 

I 

t . 

I In view of the above, it is an object of the 
20 preslent Invention to provide a method and apparatus 
for iproducing an animation of a group Including a 
plurality of members whose behavior is dynamically 
contjrolled. It is another object of the present 
inv&;ntion to provide an entertainment apparatus which 
25 uses; an animation produced by the above method. 

i According to an aspect of the present invention. 
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to Achieve the above objects, there is provided a 

method of producing an animation 6£ a group Including 

a pljurallty of members, the method comprising a 

continuous processing step for determining the states 

(such as a location and a dynamic state) of the 

.1 ■ . ■ 

respective members at each time step in accordance 

with predetermined dynamic characteristics of the 

respiective. members, and a discrete processing step for 

accepting, from the outside, a command specifying the 

oveztall state (such as a shape or a dynamic state of 

the Igroup) to be achieved for the group and assigning 

rolejs to the respective members of the group in 

aocclrdance with the overall state specified by the 

command. In the method, if new roles are assigned to 

members in the discrete processing step, the dynamic 

states of the respective members are adjusted « in the 

continuous processing step, in accordance with the 

assigned new roles, 

i According to another aspect of the present 

invention « there is provided an entertainment 

i 
I 

appalratus comprising an input unit for Inputting a 
comniand Issued by a user or an operation performed by 
a user, an executing unit for executing a prestored 
entelrtainment program in accordance with the command 
or dperation, and an output unit for outputtlng the 
resiilt of the exeaution of the entertainment program. 
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In the apparatus the executing unit produces an 
animation of a group including a plurality of roembers 
ueirig the above-described method according to the 
present invention, and outputs the produced animation 
5 to the output unit so as to display the animation. 

BRIEF DESCRIPTION OF THE DRAWINGS 
iFlg. 1 is a schematic diagram Illustrating an 
example pf an animation produced using a method of 
10 controlling the movement of members as a group 
f!S according to the present invention; 

i;0 jFig. 2 is a block diagram illustrating an 



embodiment of a system for performing optimum hybrid 
feedback control according to the present invention; 



rt'jw : 

15 jFig. 3 is a schematic diagram illustrating another 



example of an animation produced using a method of 
controlling the movement of members as a group 
0 according to the present invention; 

f*^ :Fig. 4 is a schematic diagram illustrating an 

20 example of role replacements according to the present 
invention « wherein changes in the trajectories in the 
state space are shown for a group Including three 
members; 

; Fig. 5 is a schematic diagram illustrating an 
25 exainple of role replacements according to the present 
invention; 
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fi ^ ^ diagram Illustrating an _s»^M pl^^jjJ>-a 
run-itime coordinatioiu-aitr^^ 
ju^nfiirir^tnyi^^ i i^n r 

:Flg. 7A is a graph illustrating the performance 
Index of a system according to the present invention 
ad a function of the frame number « which corresponds 
to the elapsed time, and Fig. 7B is a graph 
Illustrating the number of local interactions in the 
syst;em according to the present Invention as a 
function of the frame number, which corresponds to the 
elapsed time; 

IFig. 8 is a diagram illustrating an example of an 
algqrlthm of a version optimized in terms of a block 
diagonal system; 

:Flg. 9 is a table illustrating the evaluation 
results in terms of the stability and the calculation 
load during a game, for the algorithm according to the 
present invention; 

•Fig. 10 is a schematic diagram illustrating one 
scene of an animation of a group of bees produced 
according to the present invention; 

I.Flg. 11 illustrates examples of matrices used to 
generate the animation shown In Fig. 10; 

Fig. 12A is a graph illustrating the evaluation 
funqtlon (optimal cost) for another example of an 
animation as a function of the frame number, and Fig. 
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12B is a timing chart illustrating the changes in the 
state space in terms of the roles assigned to six 
airplanes contained in an formation of airplanes used 
in the example shown in Fig. 12A; 
5 I Fig. 13A is a graph illustrating the performance 

index associated with another example of an animation 
according to the present invention as a function of 
the frame number, and Pig. 13B is a timing chart 
illustrating the changes in the state space in terms 
10 of the roles assigned to six fishes contained in a 
group used in the example shown in Fig. 13A; 



f!:0 :Fig. 14 is a schematic diagram illustrating an 

ru : 

y example of an off-line algorithm for calculating 

!P parameters associated with a coordination process 

^'"^ 15 according to the present invention; 



j Fig. 15 is a schematic diagram illustrating 
another example of an off-line algorithm for 



0 i 

^ calculating parameters associated with a coordination 

process according to the present invention; 
20 ; Fig. 16 is a functional block diagram of an 

entertainment apparatus using a control method 
according to the present invention; 

: Fig. 17 is a schematic diagram illustrating an 
example of the hardware configuration of an 
25 entertainment apparatus using the control method 
according to the present invention; and 
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Pig. 18 is a circuit diagram illustrating. an 
example of a circuit configuration of the 
entertainment apparatus using the control method 
according to the present invention. 

DESCRIPTION OF THE PREFERRED EMBODIMENTS 
With reference to the accompanying drawings, 
embodiments of a method and apparatus for producing an 
animation of a group including a plurality of members 
whose motion is coordinated by means of dynamic 
control according to the present invention, so that 
the members behave in a desired fashion as a group, 
and also an embodiment of an entertainment apparatus 
using the method are described. 

(A) Method and Apparatus for Producing Group Animation 
: A typical example of cooperative group behavior is 
the isystematic or coordinated behavior of a group of 
fishes or animals. However, the group behavior 
appearing at the surface is not always controlled 
perfectly, as is the case in a group falling down from 
the itop of a cliff at full speed. 

In the natural world, when there is no control 
system which controls the entire group, the relative 
motion of each member of the group is tightly 
constrained by local interactions. In many cases. 
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local Interactions occur not as an objective action 
but afi a by-*product« 

In the present embodiment, by way of example « a 
group of fishes or animals Is animated by means of a 
5 dynamic simulation and on the basis of a physical 
model. Herein It is assumed that the intrinsic 
characteristics of the group are poorly controlled and 
thus the group behaves as a potentially unstable 
dynamic system. In the present embodiment, taking into 
10 account the above, the physical characteristics 

associated with the motion, of the members of the group 
are reflected in the animation. 

(A-i) Motivation for Use of Dynamic Group Motion 
15 In the present invention, the term "dynamics" is 

used to describe a general dynamic system. This is 
applied not only to a physical model or a quasi- 
physical model but also to a model which describes the 
behavior by mathematical expressions such as a 
20 combination of differential equations and finite 
difference equations Including a discrete event 
component In a finite state machine or the llke- 

! In computer animation, the application of dynamic 
characteristics eliminates a lot of tedious manual 
25 tasks required In synthesis of the motion of a group 
including a very large number of members. If the 
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motion of a group including a plurality of members is 
represented on the basis of a model rather than 
trajectories, it becomes possible to easily make 
modifications to adapt to various requirements at the 
design stage. That is, even if the environment or the 
sceiiario is modified many times, the use of a model 
allows trajectories to be easily calculated for the 
modified environment or scenario. 

In interactive animations and video games in which 
an infinite number of events can occur in response to 
unpredictable operations performed by a user, 
employment of the group animation based on the dynamic 
characteristics is very useful. In order that the 
motion of a group is controlled by a scenario or a 
game player without falling into a confused and 
uncooperative state, but intentional or forced 
coordination, rather than spontaneous coordination, 
among the group members is required. 

(At 2) Main Improvements 

If animation of a group including a plurality of 
members on the basis of independent behaviors of the 
individual members is attempted, it is difficult to 
control the group in accordance with a given scenario. 
In order to form a. group in a desired fashion with a 
required precision at a correct time, it is effective 
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to give a command to control the group ae a whole with 
coordination. To this end, the following Improvements 
are required. 

First, high-precision coordination among the 
individual trajectories is required to realise 
systematic behavior. Essentially, this is a problem of 
continuous control. To the above end, it is desirable 
to link, to a continuous world, a program or an 
inference engine operating according to rules based on 
the discrete behaviors and empirically obtained 
knowledge, more tightly than is currently linked. 
Second, problems such as instability and 
uncontrollability intrinsic to dynamic systems often 
occur in physical animations. In the present 
embodiment I in view of the above, the following 
Improvements are made. 

1. Unified Rule for Obtaining Coordinated Motion of a 
Group 

The concept of a state-space layout is Introduced 
herein to provide a unified fundamental expression to 
specify the motion or the state of a group as a whole. 
Furthermore, it Is proposed herein that when a dynamic 
state becomes diverged, a hybrid rule consisting of 
continuous and discrete processes is employed to 
efficiently control the dynamic state into a stable 
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state • 

2. Coordination Algorithm at Run Time 

We herein propose a coordination algorithm for 
controlling individual members of the group frame by 
frame instead of treating them as a mass. This 
algorithm exerts high performance in real time when 
being applied to a small group including a small 
number of members. 

3. Guideline for Adjustment of Dynamic Model 

. The optimum performance of this coordination 
algorithm is assured for linear dynamic 
characteristics. For complicated dynamic 
characteristics, a guideline is presented for 
adjustment of the dynamic model to handle nonlinearlty 
and maintain high performance « 

4. Demonstration 

Fig. 1 illustrates an example of an animation 
which has a high dynamic level and thus which could 
not be achieved by any other method because of the 
problems of instability and uncontrollability • In Fig. 
1. the result of the pioneering work by Tu and 
Tarzopoulos (X. Tu and D. Terzopoulos, "Artificial 
Fishes: Physics , Locomotion, Perception, and Behavior", 
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Proceedings of Slggraph 94« Jul., 1994, 24-29} Is 
herein reproduced by the technique according to the 
present invention. Herein, coordinated motion of a 
group id realized in an animation such that the group 
5 behaves systematically with high- accuracy dynamic 
characteristics. Thus, the group is maintained in a 
coordinated form without falling into a diverged state 
regardless of a disturbance. In the example shown in 
Fig. 1, a group of fishes having physical 
10 characteristics moves against the flow of water, and 
the group of fishes is illuminated with light. 

(A-3) Problems to be Solved 

In the present embodiment, the group coordination 

15 is separated as an intermediate layer between dynamic 
members and the interactive scenario. In the 
coordination according to the present embodiment, 
roles and control commands are issued to the 
respective members in response to a given group-level 

20 command so as to control the members to form a group 
in a desired fashion. 

! k first assumption made in the present embodiment 
is that a command specifying a group-level state can . 
be issued as a layout command wherein the layout 

25 refers to a combination of operating points in a state 
space. Before describing the concept of the layout, it 
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Is first described that a command to control members 
to form a group can be described by one of the methods 
listed below or one of various other methods; 

(a) A combination of labeled or non-labeled target 
5 locations described globally, locally, or on a frame- 
by-frame basis; 

(b) A verbatim description of the manner of 
forming a group; 

(c) Redistribution map of the member density In a 
10 particular scene; and 

(d) Specifying a dynamic level by a velocity 
vector. 

In order to Illustrate the effectiveness of the 
above methods, the manner In which the group changes 
15 in the state space with time is described below. 



Herein, the term "group" is used to describe a set of 

y 

y members whose motion variables are gathered into 

!jf corresponding state vectors, 

1'^ A member state vector x^^ with a height (vector 

20 size) of n^ includes all variables at a time step n, 
wherein the variables include a variable representing 
the : degree o£ freedom of a member 1, first derivatives 
which represent the dynamic state thereof, variables 
representing physical and quasi-physical states, and 
25 the internal state of motion. Member state vectors are 
combined together into a single state vector X„ a [x„\ 
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. . . . Xn**]^ With a height of N n^ to represent the 
state of the entire group. In this technique, the 
layout is not directly linked either to the respective 
individual members or the target layout, but the 
5 layout represents the distribution of non-labeled 
operating points for providing blind assignment to a 
group as a whole. That is, the layout is given by a 

vector X„£ » .C^saf^' ^ref*' • • • f *r©f^ 

l'^ (200 in Pig. 2). 
To realize the target layout, roles of members are 

10 assigned by a role assignment switchboard (201 in Fig. 
2) by rearranging the target and assigning roles to 
the individual members. Herein, this is represented by 
jr„-X^,^ where the N-n^ x N'tt^ matrix 7t^ represents a role 
assignment at time n, that is, a bisection between the 

15 target and the members (R. Grimaldi, "Discrete and 
combinatorial mathematics**, Addison-Wasley, New York, 
1999). 

Such a matrix expression is very convenient and 
will be used in the remaining part of the description 

20 based upon linear algebra. However, in some cases, a 
simple mapping table can function well. Therefore, in 
practical systems, the matrix expression is not 
necessarily required. For example, the target - 
assignment for members 1, 2, 3, 4, and 5 (selection of 

25 possible 51 « 120 bisections) can be described as 
follows: 
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..•(1) 

Furthermore, In the present embodiment, the 
optimum hybrid feedback problem Is solved as follows: 

(1) When a set of N members is given, the dynamic 
characteristics of the respective members and the 
conditioned responses linked to one another are 
collectively represented by a nonlinear mapping f as 
follows: 
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.-•(2) 
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(il) A layout is defindd for the group as a target 
state In the' state space X^qc m accordance with , a 
command Issued by a user or in accordance with a game 
scenario. That is, the layout is specified on -the -fly. 

(iii) A coordinated control = g(X„-%-X„^) is 
determined which allows formation of a group directed 
to the layout with the optimal performance (tradeoff 
between the time response and the energy consumed by 
the members) . This reduces to a hybrid (continuous- 
discrete) problem described below: 

(iii- a) Continuous Part: is driven toward n^'X„; 
using the control while minimizing a trade-off 
function J between particular energy and time. Instead 
of the function J representing the integral cost, an 
evaluation function in a quadratic form in the optimal 
regulator (LQ) problem may also be employed. 

(111-b) Discrete Event Fart: Replacement of roles 
assigned to members is performed if It is determined 
that role replacement of jc^ results in an improvement 
in performance (for example, a reduction in the cost 
function J), More specifically, the cost J required to 
maintain the present roles of the individual members 
and the cost J* required to replace the roles among K 
members having the lowest performance, which will be 
described later, are calculated and compared with each 
other to determine whether the replacement of roles 
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results in an improvement in performance. In the case 
where the total number N of members is small, K may be 
selected to be equal to N. However, when N is large, 
replacement of roles is performed only among K members, 
where K < N. 

In the above -described optimum hybrid feedback 
technique according to the present invention, the 
problem of the group coordination required to achieve 
group- level control is intentionally separated from a 
large number of other problems in the group animation. 
On the assumption that dynamic effects are dominant, 
the dynamic effects are formulated on the basis of the 
viewpoint of the dynamic system and the control 
thereof, and individual members are separated from a 
high-level layer (planning and recognition) widely 
used in multiagent or cooperative robotics on the 
basis of lA driving. However, note that the separation 
does not mean inhibition of use of the layer related 
to the system. 

Fig. 2 illustrates, in block diagram form, an 
example of a system for realizing the optimum hybrid 
feedback control according to the present invention. 
This system includes the role assignment switchboard 
201, a hybrid coordination processor 202, and a member 
state determinator 204. 

. In accordance with a layout 200 specifying a 
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particular group-level state which has been set in 
accordance with a request given from the outside, the 
role assignment switchboard 201 assigns roles required 
to achieve the layout 200 to the respective members. 

The hybrid coordination processor 202 includes an 
optimum continuous controlling unit 202a and . an 
optimum discrete role assigning unit 202b. The optimum 
continuous controlling unit 202a performs continuous 
feedback control, in accordance with the current group 
state, in the reference state in which the roles have 
been assigned by the role assignment switchboard 202 
to the respective members, thereby continuously 
determining the states of the respective members. The 
optimum discrete role assigning unit 202b controls the 
role assignment switchboard 201 so that the roles of 
the members are reassigned at predetermined times, in 
response to an external command, or depending upon the 
present state (snapshot) of the group, 

A member state determinator 204 determines the 
states of the respective members from a set of 
variables Ei describing the dynamic characteristics 
based on the states of the respective members 
determined by the optimum continuous controlling unit 
202a, taking into account a global disturbance and 
interactions' among the members, thereby determining 
the state of the group at the current time. 
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In the hybrid coordination process performed by 
the system shown in Fig. 2, taking into account the 
Internal dynamic characteristics of members by 
Introducing the concept of encapsulated members is 
important in designing of the feedback control. The 
coordination acts as a clutch between the interactive 
scenario and members, and the group is treated not as 
a black box but as a white box (model having a known 
structure) • 

The hybrid coordination process shown in Fig. 2 
makes it possible to realize an interface between 
discrete planning and continuous dynamic 
characteristics via coordination between discrete 
motion and continuous motion. 

In many cases, the hybrid coordination process 
according to the present invention can be 
advantageously used. Fig. 3 illustrates one example. 
In the example shown in Fig« 3< a large number of 
airplanes fly as a group, in which pilots are required 
to fly their airplanes in coordination with each other. 
In this case, because the continuous feedback control 
becomes more dominant than the discrete determinations, 
it is difficult to represent the flights on the basis 
of conventional synthesis rules. The coordinated 
flight animation shown in Fig. 3 will be described in 
further detail later. 
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(A- 4) Theoretical Background 

The present Invention is based upon the state 
space approach (disclosed In H. van de Panne, E. Flume, 
5 and Z. G. Vranesic, "Reusable Motion Synthesis Using 
State-Space Controllers", in Proceedings of SIG6RAPH 
1990, Computer Graphics Proceedings, ACM SIGGRAPH, 
1990, 225-234} and also based upon the response 
feedback control. 
10 Furthermore, a plurality of members are 

d simultaneously controlled within a linear optimal 

fig control framework (disclpsed in L. Brotman and A. 

Netravali, "Motion interpolation by optimum control". 



ru 
y 

in Proceedings of Siggraph 88, Aug. 1988. 309-315) 
15 wherein the control of the plurality of members 
includes replacement of roles assigned to the 
respective members. 
tp From a theoretical viewpoint, the method according 

|<sb to the piresent invention is characterized by the 

20 hybrid control processing consisting of time-sampled 
continuous processing and discrete event processing, 
which are combined together and related to each other. 

(Ar4-1) Hybrid System and Hybrid Control 
25 More particularly / this system is a so-called 

switched system including a group of continuous time 
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subsystems which are set In correspondence with 
specific role assignments and also Including a rule of 
coordinating the switching (role replacement) among 
the subsystems • The field of such a hybrid system Is 
Interdisciplinary and Is at the boundary between 
control technology and computer science. Applications 
of the hybrid system are expanding Into other fields 
such as automobile, electrical power supply, aircraft, 
and traffic control technologies (Special Issue on 
Hybi^ld Control Systems « IEEE Control Systems Magazine, 
19(4), August 1999) In which themes of animations of 
Interest Include a collision, traffic, etc. 

(A-4-2) Coordination of State-Space Trajectory 

Controlling a group Including a plurality of 
members such that the group has a desired form can be 
regarded as a problem of feedback control. The dynamic 
characteristics of a group Involve not only 
conditioned responses and Interactions (for example. 
Instinctive behaviors of a group) but also many other 
events related to one another. . (for example, 
controlling of machines, muscles, and motors). Such a 
system Is generally represented by a nonlinear mapping 
f with a combination of state variables. In the 
present embodiment, the group as a whole Is regarded 
as a dynamic system and formulated. The formulation is 
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then applied to the group animation. 

In the present embodiment, the formulation of the 
coordinated motion of the groiip reduces to finding an 
admissible control set {Uno-*-Uj which minimizes the 
performance index J( {U^o • • • -Uj, (X^o*--X«}) constrained 
by a particular evaluation condition or by Equation 
(3) descried below. 

' Vne[no...»l: X«,i - f(X^, UJ 
.(3) 

Although Infinity Is employed In Equation 3, the 
actual settling time (the time required for the state 
variables to settle within ±5* around the target 
values) can be generally reduced to an arbitrary value 
by Increasing the control level. In this nonlinear 
optimization problem, the processing of trajectories 
and inputting of tasks are required over the period of 

time [1- • -^^l • 

In applloatlons to Interactive animations, global 
searching is Impossible and thus it Is necessary to 
perform a process at each time step. For the above 
reason, a closed control rule = g(XJ Is required to 
convert the optimization problem into an optimum 
feedback problem, wherein g is a nonlinear mapping. 

It will be useful to discuss here the treatment of 
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nonlinearity based on linear design before discussing 
the nonlinear control (J. Slotlne and W. Li, Applied 
Nonlinear Control", Prentice Hall, 1991, 461 pages). 
This approach is well known to those skilled in the 
art, and it has been established on the basis of a 
reliable theory. For example, gain scheduling taking 
into account groups to be switched based on a linear 
model is described in D. A. Lawrence and J. Rugh, 
"Gain Scheduling Linear -Dynamic Controllers for a 
Nonlinear Plant**, (Automatic, 31(3), 1995, pp. 381-390, 
and robust control in which nonlinearity is reduced to 
a disturbance is described in K. Zhou, "Essentials of 
Robust Control", (Prentice Hall International, 1998. 

This above approach is a practical choice in the 
art of animation technology. That is, in virtual 
worlds, it is reasonable to manually build a dynamic 
model such that it behaves well. It is interesting to 
note that two physicists, Toner and Tu, who have 
proposed the mathematical theory of producing a group, 
have both employed similar two-step approaches in 
which linear approximation is performed first and then 
nonlinearity is taken into account (J. Toner and Y. Tu, 
"Flocks, herds, and schools: A Quantitative Theory of. 
Flocking", Physical Review Letters, Vol. 58, 1999). 
The extraction of nonlinearity and the separate 
treatment thereof bring the problem back to the field 
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of linear feedback control theory. Thus « It Is assumed 
herein that the dynamic characteristics of a natural 
system can be represented by a set of matrices A and B 
as described below. 

5 

. X„., = AX, + BU, 
...(4) 

where the matrix A can be represented In the following 
10 form (the matrix B can also be representing in a 
similar form) : 




...(5) 

15 

Square matrices of order of ne appearing on the 
diagonal line of the matrix A represent the dynamic 
characteristics of the respective members. On the 
other hand, elements C^^ lying off the diagonal line 
20 may be used to represent ''natural" couplings among the 
members (due to, for example, the instinct of the 
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group). Such a conditioned response is hard-wired and 
embedded at a low level in the dynamic characteristics. 

In another embodiment, elements of the matrix B 
may be used to identify a command which is issued to a 
5 single member but which can influence a plurality of 
members and to share the same motivation among 
subgroups. 

The fundamentals of linear optimization control 
and linear quadratic (LQ) synthesis (G. P. Franklin, J. 
10 . D. Powell, and M. Workman, "Digital Control of Dynamic 
i'3 Systems", J^ddison Wasley, 1998) are discussed below. 

l|5 The multivariable optimization control rule g can be 

given by a matrix gain G which minimizes a performance 
index (which will be described below) represented by 



I'U 

3 -ST 



15 the state- space trajectory of the feedback system: 

i! 

W J - - y XlQX. + UlRU, 



J - I 2 XjQX. + Mil „ 
W . . . ( 6 ) 



Because J has a quadratic form, a local 
20 minimization condition is required. This condition is 
sufficient to achieve a global minimum value. This 
evaluation criterion has a fixed structure and 
provides to a designer a means for adjusting the 
dynamic behavior by matrices Q and R while making a 
25 compromise between the response time of a selected 
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state variable and the performanoe level (energy) of a 
process. 

In the linear case, Riooati'e equation (RE) is 
obtained by solving the minimization problem under 
constrained conditions by means of Lagrange's method 
of undetermined multipliers: 

S„ » A^tS„.i-S«,,B(R+B*S„,iB)-^B^S„,JA+Q --.(7) 

In order to solve Riccati's equation, it is 
required to apply Equation (7) starting from the end 
point in a reverse direction along the time axis so as 
to obtain a time-varying control rule (G. P. Franklin, 
J. D. Powell, and M. Worman, "Digital Control of 
Dynamic Systems", Addison Wesleyr 1988). Fortunately, 
it is known that the time-varying control rule quickly 
becomes stable, and the time-varying control rule can 
be well approximated by increasing h toward oo. Thus, 
we can obtain a steady state version of Riccati's 
equation (ARE) to be considered, as described below: 

S. « A^[S.-S,B(R+B'S,B)-^B'S,]A+Q ...(8) 

It is generally impossible to obtain a solution in 
a closed form for Equation (8) except for very simple 
cases; Thus, a numerical approach is employed at this 
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stage. In Section (A-8), the eigenvalue expansion 
procedure which is widely employed in various CAD 
packages used in the design of control systems will be 
briefly described. If S. is determined, the optimum 
feedback rule is given as follows: 

. . . ( 9 ) • 

The control gain G is a perfect matrix, that is, a 
multivariable controller which performs coordination 
taking into account coupling among the members and 
which minimizes the evaluation criterion at the group 
level. In contrast, a diagonal matrix or a block 
diagonal matrix controls the respective members 
independently. 

(A-4-3) Coordinated Response to Discrete Events 

In the above discussion, it is assumed that the 
initially assigned roles are valid, that is, n, Jt^^^aXo 
over a period in which a particular layout is given. 
However, such a stable state is not realistic in time- 
varying environments which of ten occur in interactive 
games. In order to achieve effective coordination in a 
group, it is required to replace roles between members 
to compensate for various events and factors, whenever 
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there is an opportunity to perform replacement. The 
opportunity for role replacement can occur when one of 
the conditions described below is met. Note that, the 
role replacement opportunity is not limited to those 
5 described below: 

(a) Reduction in run- time performance due to a 
conflict- between the nonlinear dynamics and the linear 
assumption; 

(b) Addition or removal of a member to or from a 
10 group; which causes necessity for on-the-fly 

rearrangement of a layout; and 
ijfj (c) Occurrence of a discrete event such as a usual 

collision or non-modeled uncertainty in a time-varying 



m 

lit « 

environment 



f4. 15 In addition to the opportunities described above, 

replacement of roles may also be performed when a 

command is Issued by a user or by a scenario to 

w 

113 generate a new group, or when a change in the number 

0 • 

jf* of members occurs, or otherwise when the motion of 

20 members is affected by an obstruction or a collision. 
In practical application to an animation, it is 
required to determine the states or locations of the 
respective members frame by frame. However, the 
assigned roles may be maintained over a plurality of 
25 frames, and replacement of roles may be performed only 
when the degree of necessity for the replacement 
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becomes very high. In this case, it is required to 
strictly define the conditions which should be met to 
perform the role replacement. Furthermore, in this 
case, the evaluation or comparison of candidates to 
5 determine K members to be replaced in roles is 

inhibited or K is set to a small value over a period 
between events which can trigger the role replacement. 

The effects of replacement of roles are described 
below with reference to Fig. 4. Fig. 4 illustrates 

10 . trajectories in the state space for a group including 
three members. In Fig. 4/ when an event occurs in an 
initial state space 401 in which a target layout 410 
has been set in advance, if replacement of roles is 
performed, the state space 401 is changed to a state 

15 space 403. However, if replacement of roles is not 



0 performed, the initial state space 401 becomes equal 

1^1 to the state space 402. 

P The assignment of roles is performed in accordance 

with a state snapshot. This becomes possible because 
20 of a significantly important theorem of optimum linear 
control theory which is described below. That is, the 
cost required to drive a dynamic system into a target 
state n*Xy,f can be estimated as follows: 

25 J « (X„-yX,,jX[X,-vX„rl 

. . . (10) 
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The role replaoement , that la, alternative 
bljectlons (k)/k « 1...N! between members and the 
target can be evaluated and compared with one another 
by numerically evaluating the evaluation gains 
obtained by the respective options thereby evaluating 
the degrees of acceleration of the. convergence toward 
the specified target achieved by the respective 
options. This is illustrated in further detail below 
with reference to Fig. 5. 

In the exeunple shown in Fig. 5, the members are 

ranked in accordance with the ease of reaching the 

targets of the members by estimating the integral cost 

Ji - i< - ^.J Qi< - ^r.J + iO' RuJ^ Herein, J, is 

approximated by ''^ " " *^i«<^' ^- ~ ^xut^ if s. is 
replaced with an identity matrix « the result decreases 

to a Euclidean distance. However, the reduction in 

distance does not always result in an Increase in the 

degree of ease of reaching targets. For example, 

although fishes 4 and 5 are near the targets, they 

have to change their direction of motion into a 

lateral direction to reach the targets. Note that 

updating of jr^ is performed in a rather shortsighted 

fashion and the best option is immediately employed. 

(A-S) Algorithm 
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An example of a novel algorithm for achieving 
coordination at run time according to the. present 
invention is shown in Fig. 6. This example of the 
algorithm shown in Fig. 6 has a high responsiveness 
and may be performed in real time. Furthermore, at the 
design stage, the optimum feedback rule is determined 
from the characteristics of the dynamic system 
according to the evaluation criteria using an off-line 
algorithm including a procedure widely used to solve 
the LQ problem. This will be discussed in further 
detail later in Section (A-8). 

The algorithm shown in Fig. 6 is applied for each 
time step. The calculation efficiency of the algorithm 
may be increased by various methods including a 
clustering technique used in a simulation of a many^ 
body (n-body) problem or a method adapted to 
characteristics intrinsic to a particular case. 
Although a matrix representation is employed, the 
matrix representation lis not always efficient. 

Because the complexity of the role assignment 
increases as the factorial of the number of members, 
the: above algorithm cannot be scaled up unless the 
role replacement is limited to a small subgroup. 

In essence, K members having the lowest 
performance (worst performers) and thus having a large 
influence upon the overall performance index J are 
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tracked so that when an opportunity for role 
replacement Is found, role replacement Is performed 
among those worst performers, thereby reducing the 
calculation load from N! to Kl (K < N or K << N) . When 
the calculation Is performed in real time, K generally 
has a small value such as 6 • For example , K and N may 
be set such that K and N » 100. When N Is small , an 
Initial sorting process Is skipped, and thus a 
corresponding overhead Is eliminated. 



(A- 5-1) Modification Associated with Independent 



When coupling among members Is negligible. 



llQ Member Dynamloa 

separate optimum controllers (which may be obtained by 
|,i 15 means of LQ synthesis) are used for the respective 



members ( j ) . The contribution to the overall 
performance is given by J{j) » [X„^ - («r'X„JM^S**"'"^[X,3 
0 - (J^n'X^ef)^] Where (ji^-X^^J^ represents the Jth coordinate 

j.^ of the current reference vector Jth'^rof Note that the 

20 role assignment and the layout are maintained at n„ and 
Xye£# respectively. By comparing not the absolute cost 
but. the derivative thereof and employing factorization 
in the calculation, the complexity of J Is reduced 
from 0(N*) to 0(N). 



25 



(A- 5 -2) Convergence Analysis 
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The control theory includes a tool for evaluating 
the dynamic behaviors . This tool is useful to check 
whether the specified layout is maintained. If the 
dynamic characteristics based on which the control is 
performed are linear, it is possible to obtain formal 
proof of convergence from the essential stability of 
the LQ control (F. Franklin, J.D. Powell, and M. 
Workman, "Digital Control of Dynamic Systems", Addison 
Wesley, 1998). According to Liapunov's method (J* La 
Salle, J., and S. Leffschetz, "Stability by Llapunov's 
Direct Method", Academic Press, New York, 1961), the 
system is globally stable when the Llapunov function 
satisfies the following conditions: 

1. V(Xn) Is continuous and positive. 

2. V(X„) decreases with time. 

3. There is a function of (X„) which satisfies the 
following conditions: 

(a) 0 < (I |XJ I) < V(XJ 

(b) lim ( I |XJ I )- 00 when | |xj | oq 

Because the stability of the LQ problem is ensured 
(6. F. Franklin, J. D. Powell, and M. Workman, 
"Digital Control of Dynamic Systems", Addison Wesley, 

1998), a Llapunov function V(XJ = XjS«X„ which 
satisfies the conditions 1 and 3 described above is 
selected. 
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...(11) 



Herein 



, S. = A'[S, -S^,iB(R+B'S,,iB)-^BX,JA Q, and 



thus V « -Xn*[Q + G*RG1X„. Herein, IQ + G'^RGl is 
positive and finite, hence V < 0. Thid ensures the 



convergence toward the designated layout is assured. 
In the present case* the replacement of roles 
(permutation it^) provides an opportunity for a further 
reduction in the calculation load, and thus the 
convergence is accelerated « 

V(jt,X„.x)<V(X„,J V(Ji„X,,J^V(X„.i)<0 
..•(12) 

Equation 12 indicates that the gain can be further 
increased by optimising the time sequence {jCq, . . . , Jt„, 
nn+i' associated with the assigned roles. 

Herein, an instantaneous optimum choice (which may be 
less optimum than the really optimum choice because of 
the shortsighted approach) is employed taking into 
account the application. Furthermore, complex 
switching is prevented using a threshold value A^iin (a 



stability. That is, when assigned roles are fixed. 



SC1626 

recent discussion on the effect of switching rules 
upon the stability can he found In D. Llberzon and A. 
Morese, "Basic Problems In Stability and Design of 
Switched Systems" , IEEE Control Systems Magazine, Oct. 
5 1999). 

Although Llapunov's method can be applied to 
nonlinear dynamics, the execution thereof is 
complicated and It is necessary to perform a computer 
simulation to check whether the algorithm is trapped. 

10 Fig. 7A is a graph showing the performance index, and 
Fig. 7B is a graph showing the number of local 
interactions. In Fig. 7B showing the number of local 
interactions . open bars, represent the number of 
collisions and isolid bars represent the number of role 

15 replacements. In both Figs. 7A and 7B, the horizontal 
axis represents the frame number which corresponds to 
the elapsed time. It can be seen from Figs. 7A and 7B 
that the number of collisions and the performance 
index both decrease with increasing frame number and 

20 with increasing number of role replacements. 

(A^5-3) Real-Time Performance 

The algorithm (Figs. 6 and 8) according to the 
present invention has been evaluated in terms of the 
. 25 stability and the calculation load imposed upon a game 
under the conditions of a frame rate of 50 Hz and. a 
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peak number of instructions of the order of GFLOPS/sec 
(Flg« 9). The algorithm has been executed on the 
PlayStation 2 produced by Sony Computer Entertainment, 
Inc. It has been assumed that the CPU time spent for 
5 the processing associated with the dynamic 

characteristics and the coordination Is very short (of 
the order 1% of the total CPU time), and the number of 
members whose roles were replaced was limited to K = 6. 
When the number N of members is small, the processing 
10 associated with the coordination and the processing 
associated with the dynamic characteristic need 
lio similar calculation loads. However, the processing 

associated with the coordination becomes dominant when 



*f N Is large. 



w 

Ass. 



16 The algorithm A2 shown in Fig. 8 Is an example o£ 



'^'^ an algorithm of a version optimized In terms of the 



block diagonal system. . The algorithm A2 shown in Fig. 



¥ 8 has better linear characteristics than the algorithm 

0- . 

Al shown in Pig. 6, because the core matrices S,, A, B, 
20 and 6 in the algorithm A2 increase in size linearly as 

N Increases, while they Increase in size quadratically 

in the algorithm Al. 

The discrete part in the coordination process in 

the algorithm A2 may be treated as an ordinal 
25 assignment problem according to a technique known In 

the art of operational research. For example, when K 
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becomes large « a streamlined version of the simplex 
method (linear programming) known as the 
transportation simplex method or a known algorithm 
developed for assignment problems In the art of 
5 operational research may be employed, Instead of the 
above-described process of replacing roles using a 
permutation table. 

(A- 6) Practical Use of the Algorithm 
10 In order to employ the theory according to the 

Q present invention in practical applications, various 

associated techniques including a modeling technique. 



a simulation technique, a content authoring tool, a 



run-time engine, etc,, have been developed. The 
developed prototype is based on 
Matlab, Maya, and PlayStation 2. 



•r 

1,^^ . 15 developed prototype is based on the run- time codes of 



(A-6<-l) Discussion on Nonlinearity 

The efficiency of the coordination process depends 

20 upon the nonlinearity, that is, upon whether the 

dynamic model, which would be built manually in the 
conventional technique, can be well processed using 
linear feedback, because global optimization 1& 
assured only for linear dynamic characteristics. 

25 The computerlza.tlon of such a task is not easy, 

and specialized knowledge about dynamic 
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characteristics is required. Some practical guidelines 
are presented below. 



including 50 members (bees) is described. In this 
example « the bees in the group behave as a group (have 
linear attraction), while they behave repulsively when 
they come very close to each other (that is, there is 

10 nonlinearlty and the Newton's law of collision holds). 
One scene of an animation of the group of bees is 
shown in Fig. 10 « 

In Fig. 10, 50 bees gather within a particular 
region below light. Although a dynamic model employed 

15 herein is not a mechanical model employed in many 
other cases, the dynamic model employed allows 
suppression of acceleration of expansion of members. 
Thus, a certain systematic group behavior is described 
by this dynamic model. 

20 The system equation is given as X^.x = AX„ + BU„ + 

(X„). Matrices such as those illustrated conceptually 
in Fig. 11 are required for the calculation according 
to the algorithm Al. In Fig. 11, the matrix G is used 
to provide inputs of members and the matrix S is used 

25 to determine the cost (performance Index) such as a 
simple distance. In Fig. 11, non-zero elements in the 



5 



(A. 6- 1.1) Neglecting Nonlinearlty 

By way of example, an animation of a group 
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matrices are represented by shaded areas, and zero 
elements are represented by open areas. For a group 
which includes a large number of members, as is the 
case with the present group of bees, the calculation 
5 load associated with the multivariable feedback 

becomes greater than that of the simulation. This can 
also be seen from comparison between the sizes of 
matrices A and B and the sizes of matrices 6 and S.. 
For the compensation of the group behavior (for a 

10 constant disturbance), a feedforward gain (D. Powell 
and M. Workman, "Digital Control of Dynamic Systems", 
Addison Wesley, 1998) is used. First, the optimum 
feedback is designed from A, B, Q, and R on the basis 
of the LQ synthesis- The effects thereof are then 

15 evaluated by means of a simulation. 

As shown in Fig. 7, although the group is 
collapsed at an initial stage, bees gradually gather 
into a specified form after a layout is applied when 
the frame number « 21. In the present example, the 

20 number of role replacements is limited to 6 for each 
frame . Because the replacement of roles needs a finite 
period of time, a greater number of members of the 
group can be involved in practice. In spite of 
collisions, the group of bees forms a specified shape. 

25 

(A-6-1-2) Deductive Approach to Preventing 



42 



SC1626 

Nonlinearity 

A second example Is now shown to discuss a group 
of airplanes having nonlinear characteristics. To 
handle the present problem, linearity is forcedly 
5 introduced by using a Jacobian which linearly 
simulates the dynamic behavior around a certain 
operating point in a state space. Upon the assumption 
that airplanes fly in formation at a substantially 
constant speed, the Jacobian of the system is 
10 extracted? 
ID - . ' 

S ax„,, » of/ax),ax^+{af/au)o5u,-A8x^+B6u„ 



. . . (13) 



15 Fig. 12A is a graph illustrating the evaluation 



function (optimal cost) for the present example as a 
function of the frame number. Fig. 12B is a timing 



W 

ii3 chart illustrating the changes in the state space in 



terms of the roles assigned to six airplanes contained 
20 in the airplane formation of the present example. In 
the present example, the control rule is calculated 
upon the assumption that airplanes fly in formation 
and an operating point is obtained near the point at 
which the Jacobian of the system is extracted. The 
25 layout is changed at frame number lOO and also at 
frame number « 200 thereby triggering the replacement 
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of roles among airplanes. 

As can be seen from Pig. 12, when the nonllnearity 
is not very strong, as is the case in the present 
example, the performance index generally reduces. The 
5 threshold value is reduced to about 30% in order to 
avoid flicker and sporadic replacement of roles, 

(A-6-^1-3) Inevitable Handling of Nonlinearlty 

A third example is now presented to discuss a 

10 group consisting of six fishes, having (I) caudal fins 
providing a propulsive force, (II) a nonlinear 
combination of state variables in a hydrodynamic 
system, and (III) a plurality of nonlinoarities caused 
by conditioned responses to avoid collisions. 

15 When fishes of the group move against the flow of 

water, an operating point is presented. The Jacobian 
is extracted in the vicinity of the operating point. 
The caudal fins are moved in response to an impulse 
train triggered by a negative-phase signal output by a 

20 clock generator. The motion of the caudal fins (F', P' 
) can be controlled over each time -average driving 
cycle, as pointed out by Nougaret et al (J.I. Nougaret, 
B. Arnaldi, and P. Multon, "Coarse to Pine Design of 
Feedback Controllers for Dynamic Locomotion", Visual 

25 Computer, 4(10), 1997). 

The nonllnearlties cause the state- space 
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trajectories of fishes o£ the group to become 
complicated, as shown in Pig. 13, and the algorithm 
functions in a manner which is very different from the 
predicted manner. 
5 Pig. 13A illustrates the performance index in the 

present example as a function of the frame number. Fig. 
13B la a timing chart illustrating the changes in the 
state space in terms of the roles assigned to six 
fishes contained in the group in the present example. 
10 wherein lines represent respective fishes and the 

vertical coordinate represents targets. When there is 
strong nonlihearity as is the case in the present 
example, the performance index decreases in a random 
fashion. For example, replacement of roles is 
15 triggered by a change in. the layout at frame number » 



if 

^ 300. Spontaneous replacement of roles occurs near 

j^'l frame number « 200. 

«.3 In some cases, the intrinsic dynamic 

characteristics preclude the above-described motion or 
20 global searching is essentially required. In such a 
case* it often becomes impossible to reach the 
specified layout. In order to adjust the performance 
of the algorithm to avoid the above problem, it is 
necessary to perform a numerical simulation which 
25 needs a very large number of calculations. 
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(A- 6 -2) Limitations and Remaining Problems to be 
Solvad 

The algorithm which has been described above with 
reference to specific examples of applications has 
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been proposed herein in view of the known techniques 
incapable of efficiently controlling a dynamic group 
including a plurality of members while maintaining the 
trajectories in stable states even in the presence of 
an external disturbance. However. If uncompensated 
nonlinearity is carelessly introduced into the system, 
the approach according to the present invention can be 
defeated. From this point of view, an improvement in 
the robustness of the system is required. 

For example, in the present Invention, collisions 
are avoided in a modest manner by adding a dynamic 
equation which causes interacting members of the group 
to be coupled (temporarily). However, J is evaluated 
without talcing into account whether the resultant path 
includes a collision. If a linear repulsive function 
is used to avoid collisions . an optimum path including 
no collisions may be obtained. However, In this case, 
a reduction in the performance occurs. A layout may be 
issued in accordance with a sequence so as to reduce 
the possibility of collisions. However, a numerical 
simulation is always required to check the convergence 
and to adjust the threshold parameter A„v„. 

When convergence is not assured, the implicit 
nonlinear dynamics can make reaching the specified 
layout impossible. A possible eventual means is to try 
to handle the trajectories at time Intervals using a 
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nonlinear optimization algorithm. However, this 
approach precludes the interactions. Thus, the object 
to be achieved next is to enhance the robustness of 
the state feedback by fostering a better understanding 
of the hybrid system and the robustness thereof- The 
incorporation of a better collision avoidance rule is 
also required to enhance the robustness of the 
algorithm. 

(A- 7) Conclusions 

A technique has been described above for 
automatically Incorporating the control rule analyzing 
the behavior thereby controlling the group in a 
coordinated fashion within the framework of optimum 
feedback. The dynamic characteristics which need 
continuous feedback for stabilization have also been 
discussed. However, unfortunately, the approach needs 
a lot of special knowledge about the dynamics, and 
thus this technique is not universal. 

Thus, in order to release game designers from 
complicated and troublesome tasks, it is essentially 
necessary to develop workflows, software tools, and 
components. It is possible to reduce the calculation 
load by processing the complicated dynamic 
characteristics in a sequential or parallel fashion 
using the concept of the switched system. Finally, 



49 



SC1626 

planning and arguments of semantic representations of 
scenes and heuristic approaches will make it possible 
to improve the role assignment technique. 

(A-8) Off -Line Algorithm 

At the design stage, the coordination method is 
described by parameters using a matrix obtained by two 
off-line algorithms. A fir&t algorithm (Fig. 14} 
calculates the multivariable optimum feedback on the 
basis of eigenvector expansion and solves the ARE . 
(standard algorithm in LQ synthesis) 

The second off-line algorithm (Fig. 15) 
recursively generates Kl permutations and stores them 
in advance. These calculation results are sequentia:ily 
used at run time to evaluate the role replacement. 
When K = 7, the number of entries of the table becomes 
4,320. The permutation algorithm is described in R. 
Grimaldl, "Discrete an Combinational Mathematics", 
Addison-Wesley, New York, 1999. 

(B) Entertainment Apparatus 

An embodiment of an entertainment apparatus using 
an animation generated by the control method according 
to the present invention is described below. 

(B-1) Functional Block Configuration 
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Pig, 16 Illustrates, in functional block diagram 
form, main parts of the entertainment apparatus of the 
present embodiment . 

In Pig. 16, reference numeral 101 denotes control 
means used by a game player to input various commands. 
Command data input via the control means 101 is 
transmitted to game executing means 102. 

The game executing means 102 includes role playing 
game executing means 103 for executing a game in 
accordance with a predetermined scenario, group 
animation control means 104 for producing various 
groups during the game in accordance with a command 
given by the game player or in accordance with the 
scenario and controlling the produced group, and image 
generation means 105. The game executing means 102 
executes the role playing game in accordance with a 
command given by the game player via the control means 
101 and also in accordance with the game program and 
game data stored in storage means 106. Furthermore, 
the game executing means 102 produces an image using 
the image generation means 105 and displays it on 
display means 107. 

The game executing means 102 may be realized with 
hardware such as a CPU (central processing unit) and a 
GPU (graphic processing unit ) . The storage means 106 
may be realized with hardware such as a CD-ROM 
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(compact disk-read only memory), a hard disk, memory, 
or a DVD (digital video disk). 

The display means 107 may be realized with a 
monitor using a CRT (cathode ray tube) or a television 
set. The display means 107 may be combined together 
with or separated from the main unit of the video game 
machine, depending on the type of the game machine, 
that is, depending upon whether the game machine is 
for home use or for business use. 

Although in the present embodiment, the method of 
controlling a group animation according to the present 
invention is applied to the role playing game, the 
application of the present invention is not limited to 
such an embodiment. The present invention may be 
applied to various games in which a group including a 
plurality of members appears, if the dynamic 
characteristics of the respective members can be 
represented in an approximate fashion. For example, 
the present invention may be applied to a war 
simulation game and a race simulation game in which 
groups of human begins and groups of machines appear 
in various forms and move, and a game including 
artificial, organisms simulating natural organisms 
which gather into a group and behave as a group. 

(B-2) Hardware Configuration 



52 



SC1626 

An example of the appearance and an example of the 
general hardware configuration of the entertainment 
apparatus of the present embodiment are described 
below with reference to Figs. 17 and 18. 

Pig. 17 illustrates an example of the appearance 
of the entertainment apparatus- The entertainment 
apparatus 1 reads a geime program stored on an optical 
disk such as a CD-ROM and executes it in response to 
an operation performed by a game player- 

The main unit 2 of the entertainment apparatus 1 
includes a disk loading part 3 in which an optical 
disk on which a game program is stored is loaded « a 
reset switch 4 for resetting the game, a power switch 
5, a disk control switch 6 for controlling the loading 
of the optical disk, and slots 7 A and 7B (two slots in 
this specific example). 

The slots 7A and 7B are used to connect a 
controller 20- The slots 7A and 7B may also be used to 
connect a memory card so that game data is stored to 
or read from it. 

The entertainment apparatus 1 is connected to a 
monitor 30 and loudspeakers 40, although connecting 
parts are not shown in Fig. 17- In the example shown 
in Fig. 17, a television set is used as the monitor 30 
and also as the loudspeakers 40 « 

Pig. IB illustrates the general hardware 
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configuration of the entertainment apparatus 1. As 
shown in Pig. X8, the entertainment apparatus 1 
includes a control system 50 including a CPU 51 and 
associated peripheral devices, a graphic system 60 
including a GPU 62 which writes an image Into a frame 
buffer 63, a sound system 70 including an SPU (sound 
processing unit) 71 for producing musical sounds or 
sound effects, an optical disk control system 80 for 
controlling the optical disk on which the game program 
is stored, and a communication control system 90 for 
controlling the Inputting of a signal output by the 
controller 20 in response to a command given by a game 
player and also for controlling the 

inputting/outputting of data to/from a memory card 10 
in which settings of the game or the like are stored. 

In Fig. 18, the control system 50 includes the CPU 
51, a peripheral device controller 52 for controlling 
interrupt handling and direct memory access transfer 
operations, a random access memory (RAM) serving as a 
main memory 53, and a read only memory (ROM) 54 in 
which is stored an operating system (OS) for 
controlling various parts such as the main memory S3, 
the graphic system 60, and the sound system 70. 

When electric power is turned on, the CPU 51 
executes the operating system stored in the ROM 54 so 
as to control the entire entertainment apparatus l 
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including the graphic system 60, the sound system 70, 
etc. 

After executing the OS, the CPU 51 performs 
initialization and checks whether various parts work 
correctly. The CPU 51 then controls the optical disk 
control system 80 and executes the game program stored 
on the optical disk. During the execution of the game 
program, in response to a command given by the game 
player, the CPU 51 controls the graphic system 60 and 
the sound system 70 so as to display an image and 
generate a sound such as musical sounds or sound 
effects. 

In Fig. 18, the graphic system 60 includes a GTE 
(geometry transfer engine) 61 for performing a process 
such as a coordinate conversion, a GPU 62 for writing 
an image in accordance with an image write command 
given by the CPU 51, a frame buffer 63 for storing the 
image written by the GPU 62, and an image decoder 64 
for decoding image data coded in a compressed fashion. 

The GTE 61 has a parallel processing capability 
for performing a plurality of calculations in parallel. 
This allows the GTE 62 to perform, at a high speed, 
various calculations including a coordinate 
transformation, a perspective transformation, and 
lighting calculations in response to calculation 
commands given by the CPU 51, The GPU 62 writes an 
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Image composed of polygons into the frame buffer 63 
according to an image write command given by the CPU 
51. 

The frame buffer 63 is composed of a dual port RAM. 
5 and is capable of simultaneously reading data to be 
displayed and storing an image written by the GPU 62 
or transferring data from the main memory. The frame 
buffer 63 includes a display area for storing video 
data to be displayed on the monitor 30. The frame 
10 buffer 63 further includes a GLUT area for storing a 
color look-up table (CLUT) which is referred to when 
(15 the GPU 62 writes a polygon image, and a texture area 

for storing texture data which is inserted, after 
being transformed in terms of coordinates, into 
i"* 15 polygons written by the GPU 62. 

The image decoder 6.4 decodes, under the control of 
the CPU 51, image data such as still image data or 
moving image data stored in the main memory 53 and 
jiA restores the decoded image data in the main memory 53. 

20 The decoded image data may also be stored by the GPU 
62 into the frame buffer 63 so as to represent a 
background image. 

In Fig. 18, the sound system 70 includes the SPU 
71 for producing sounds such as musical sounds or 
25 sound effects in accordance with a command given by 
the CPU 51. and also includes a sound buffer 72 used 
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by the SPU 71 to store waveform data. The musloal 
sounds or sound effects produced by the SPU 71 Is 
output to the loudspeakers 40. The SPU 72 further has 
the capability of decoding audio data and also the 
5 capability of reproducing, directly or after 
modulating, waveform data. 

In Fig. 18, the optical disk control system 80 
includes an optical disk drive 81 for reading the game 
program or data stored on the opticfal disk, a decoder 
10 . 82 for decoding the game program or the data, and a 
buffer 83 for temporarily storing data read by the 
J optical disk drive 81 thereby allowing data to be read 

from the optical disk at a higher speed. 

Furthermore, in Fig. 18, the communication control 
F 15 system 90 includes a communication controller 91 for 
controlling the communication with the CPU 51 via a 
bus. The communication controller 91 Includes a 
controller connecting part 9 for the connection with 
the controller 20 used by a user to input a command 
20 and also includes a memory card insertion part 8 for 
the connection, with the memory card 10 for storing the 
game settings data or the like. 

The cohtroller 20 connected to the controller 
connection part 9 includes a plurality of control keys 
25 used by the game player to input commands. In response 
to a command from the communication controller 91, the 
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controller 20 transmits control Jcey status data. The 
communication controller 91 transfers the control key 
status data received from the controller 20 to the CPU 
51- Thus, the command issued by the game player is 
input to the CPU 51, The CPU 51 performs a process 
corresponding to the command given by the game player, 
in accordance with the game program being executed. 

When it is required to store the settings data or 
the like of the game being executed, the CPU 51 
transmits the data to be stored to the communication 
controller 91 which in turn writes the data received 
from the CPU 51 into the memory card 10 loaded in the 
slot of the memory card insertion part 8. 

One example of the entertainment apparatus and one 
example of the general hardware configuration thereof 
have been descried above. However, the present 
invention is not limited to the details of those. 
Moreover, the embodiment of the present invention can 
be carried out not only by the entertainment apparatus 
but also by a general -purpose information processor. 

For example, when a group animation is displayed 
on the entertainment apparatus using the method of 
controlling the group animation according to the 
present invention, a control key for issuing a group- 
level command to create a group or to specify the form 
of the group may be disposed, in addition to the usual 
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control keys, on the controller 20, 

The method of producing a group animation 
according to the present invention and the 
entertainment apparatus using this method have been 
5 described above. The present invention may also be 
employed to control the motion of a system including a 
plurality of Interchangeable members or objects whose 
dynamic characteristics are simulated. 

For example, the present invention may also be 
10 applied to a method of controlling a system for 
0 simulating a group including a plurality of 

1)0 interchangeable members, the method comprising a 

til 

jy continuous processing step for determining the states 

of the respective members at each time step in 
15 accordance with predetermined dynamic characteristics 



?1p 
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p of the respective members; and a discrete processing 



step for accepting, from the outside, a command 



w 

to specifying the overall state to be achieved for the 

group and assigning roles to the respective members of 
20 the group in accordance with the overall state 

specified by the command. If new roles are assigned to 
members in the discrete processing step, the dynamic 
states of the respective members are adjusted, in the 
continuous processing step, in accordance with the 
25 assigned new roles. 

Furthermore, the present invention may also be 
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applied to a method of. In a simulation of a group 
Including N (Integer equal to or greater than 2) 
members, controlling the movement of members of the 
group 60 that the members are moved from locations In 
a predetermined initial layout in a state space to 
locations In a target layout. This method comprises 
the steps of: 

a) assigning locations In the target layout to the 
respective members lying at locations in the initial 
layout so that the members are moved along the 
shortest distances to the locations in the target 
layout; 

b) moving the members in the initial layout in 
accordance with the assignments made in step a); 

c) calculating the value of a predetermined 
evaluation function associated with the movements, 
accomplished in step b), of the respective members to 
the assigned locations in the target layout; 

d) selecting K (Integer equal to or smaller than 
N) members having the greatest values of the 
evaluation function; and 

e) replacing the assignments of the locations in 
the target layout within Kl combinations of the 
selected K members. 

After completion of step e), the method returns to 
step b) so as to perform steps b) to e) repeatedly. 
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